Skip to content

Conversation

@austin3dickey
Copy link
Contributor

This PR changes go_bench_adapt.py to be more accurate about reporting GitHub information to Conbench from a local computer.

In most cases, this script will be run on the master branch during this workflow. This PR shouldn't change that use case.

However, rarely, someone might run benchmarks locally on a branch on their computer. If that's the case, we want the GitHub information to be scraped from the local repo, instead of assuming it's on the master branch. This is accomplished by passing None to github.

SCRIPTS_PATH = ARROW_ROOT / "ci" / "scripts"
RUN_REASON = "commit" if os.environ.get("CONBENCH_REF") == "master" else "branch"

if os.environ.get("CONBENCH_REF") == "master":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is CONBENCH_REF set in the workflow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, here:

CONBENCH_REF: ${{ github.ref_name }}

@austin3dickey
Copy link
Contributor Author

build is failing because of #14949

Copy link
Member

@zeroshade zeroshade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zeroshade zeroshade merged commit 885e050 into apache:master Dec 14, 2022
@kou
Copy link
Member

kou commented Dec 15, 2022

@austin3dickey @zeroshade Benchmark CI is still failing: https://github.com/apache/arrow/actions/runs/3699179581/jobs/6266261796#step:7:1212

*** Test killed with quit: ran too long (11m0s).

Could you fix it (or remove benchmark job in master) before improving Go benchmark?

Duplicated: #14921 (comment)

zeroshade added a commit that referenced this pull request Dec 15, 2022
As mentioned by @ kou here: #14952 (comment) it looks like we crossed a threshold on the Go benchmarks and have added enough benchmarks for the Compute module that they are collectively taking more than 10m to run (which is the default timeout). So this updates the timeout used for running the benchmarks to 20 minutes.

CC @ austin3dickey 

Authored-by: Matt Topol <[email protected]>
Signed-off-by: Matt Topol <[email protected]>
@ursabot
Copy link

ursabot commented Dec 16, 2022

Benchmark runs are scheduled for baseline = 6724fc1 and contender = 885e050. 885e050 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.99% ⬆️0.03%] test-mac-arm
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.58% ⬆️0.03%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 885e050b ec2-t3-xlarge-us-east-2
[Failed] 885e050b test-mac-arm
[Finished] 885e050b ursa-i9-9960x
[Finished] 885e050b ursa-thinkcentre-m75q
[Finished] 6724fc1d ec2-t3-xlarge-us-east-2
[Failed] 6724fc1d test-mac-arm
[Finished] 6724fc1d ursa-i9-9960x
[Finished] 6724fc1d ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants